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TWENTY-EIGHT SS (SUBJECTS) WERE PRESENTED WITH 
COMPUTER-GENERATED AGRAMMATICAL STATEMENTS. AND ASKED TO 
CARRY OUT TWO TASKS ON EACH OF TWO EXPERIMENTAL DAYS. TASK 1 
WAS A FORCED-CHOICE EXPERIMENT IN WHICH 50 PAIRS OF 
STATEMENTS WERE PRESENTED AURALLY TO EACH S. WHO HAD TO 
SELECT THAT MEMBER OF THE PAIR WHICH HE FELT WAS THE BEST 
APPROXIMATION TO A GOOD ENGLISH SENTENCE. IN TASK 2. SS WERE 
REQUIRED TO READ AND RANK EACH STATEMENT ON A SCALE RUNNING 
FROM ONE (COMPLETELY UNACCEPTABLE) TO FIVE (COMPLETELY 
ACCEPTABLE) • A DIFFERENT ORDER OF STIMULUS PRESENTATION WAS 
EMPLOYED ON EACH EXPERIMENTAL DAY— 14 SS WERE ASSIGNED TO ONE 
ORDER ON DAY 1. AND RECEIVED THE OTHER ORDER ON DAY 2. 

RESULTS SHOW THAT SS TEND TO PREFER THE SAME STATEMENT OVER 
ORDERS. AND THAT RANK AND PREFERENCE ARE HIGHLY CORRELATED. 
THERE ARE CONSIDERABLE DIFFERENCES IN PREFERENCE AMONG THE 50 
PAIRS OF STIMULUS ITEMS. PRE’ I MINARY ANALYSIS OF THE DATA 
SUGGESTS THAT THIS TASK MAY YIELD INFORMATION RELEVANT TO THE 
LINGUISTIC AND IN PARTICULAR THE SURFACE SYNTACTIC COMPETENCE 
OF SS WHEN APPLIED TO AGRAMMATICAL STATEMENTS. SS APPEAR TO 
BE TRYING TO COPE WITH THE STATEMENTS BY COMPARING THEM TO 
ACCEPTABLE SEMANTICAL SYNTACTIC PATTERNS. THIS REPORT APPEARS 
IN • STUDIES IN LANGUAGE AND LANGUAGE BEHAVIOR. PROGRESS 
REPORT V," SEPTEMBER 1. 1967. (AUTHOR/AMM) 
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Abstract 



28 jSs were presented with computer-generated agrammatical 
statements , and. asked to carry out 2 tasks on each of 2 experi- 
mental days. Task 1 was a forced-choice experiment in which 50 
pairs of statements were presented aurally to each S, who had to 
select that member of the pair which he felt was the best approxir- 
mation to a good English sentence. In Task 2, Ss were requlred to 
read and rank each statement on a scale running from 1 (completely 
unacceptable) to 5 (completely acceptable). A. different prider o^.l 
stimulus presentation was employed on each experimental dayj 14 Ss 
were assigned -to 1 order on Day 1, and received the other order on 
Day 2; . ■ . V‘ . *i' 
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, Results show that Ss tend to prefer the same statement over 
orders, and that rank and preference are highly correlated. There 
are considerable differences in preference among the 50 pairs of 
stimulus items. Preliminary analysis of the data suggests that r 
this task may yield information relevant to the linguistic ahi in 
particular the surface syntactic competence of Ss when applied to 
agrammatical statements, jSs appear to be trying to cope with the', 
statements by comparing them to acceptable semantical syntactic: V 
patterns. ’ . • - 
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This study examines the ability of native speakers of English to make 
judgments concerning the "well-formedness , ” in varying degrees, of random 
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strings of English sentence-format ives which are presented as putative .sentencl^S?' 
The following assumptions were made with respect to the knowledge a normal 



adult native speaker has, or has learned, about his language. 
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Elements. Linguistically, a given individual has available to him a stock 









of elements which are re-comb inable into novel sequences, and which . can be so 






re-combined readily in response to a variety of situations. However they are 



learned and fragmented from larger-to-smaller yncuttable units, this set of 
elements comprised the lexicon of the speaker. 
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Tikofsky , 

Two or more Individuals may be said to belong to the same speech community . 

' . * ’ . * - ‘ t * f <y J&jfXk 

when they possess (at least) a communality of lexicon, i.e. , when the stock of - 

' ■• f ' ; v y - * > v * j 'V ? ', *'* ' ''*V’ ' ?v^[Ss*.v ? 

elements which each possesses overlaps with another's stock of elements,; ,ae^' 
that mutual Intelligibility is possible. Overlap may be of several sorts 
c.tsnce over lap — with numbers usually small and mutual intelligibility, fc'&f 

for this small set of isolated elements, near zero ; total overlap— •in ,< the case. 

■ - * | - * • ' t t ' 1 - ^ - 4 *\ 

in which two individuals have identical histories of language. acquisifciph*; -SX 

identical habits of usage, etc.; natural overlap— -with numbers extremely Xargie 

and mutual intelligibility nearly total, even to the extent that two 'lttd'i%l^ualev^^v-r 

■ • '■ * •« • ,' ' 2f\ ■' 

may speak with one 
they do not have in common. 

Representation of elements . Each (normal-speaking) individual ha <3 the 



another for a lifetime and not discover any elemenit^ith 

■ -- --• ^ :. .■-•{•:' Wr%.:-^vW0$ 

i common. - v '- , . : ( :* 
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Lduai has,i^::-:^S£gi 

ability to represent the elements of his lexicon either phonologf cally ^ 
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observable representation is phonetic) or orthographically (if an 



available)-, or both. • ‘ s "-„ •“'• / '*.• .. sy '■*■>!<*& * -**'"-■* 

;• ' * • - .A':-'' - ,: » ';•- HW.MifM 

There is also a communality of element-represen^ ition: since nc^' 66p’ 



uals have identical pronunciation habits or handwriting, there are necessarily 
features of such representation which must be minimally, present, .in 







another to receive and respond to them. Here, also, there can exist' a 
of degrees of convergence, ranging from chance to total,.. analogously tb' : 



overlap mentioned above, with natural convergence somewhere in 
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case of natural convergence, representational skills in each individual' overlap , - 
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notice of how .the> other .acc^plisnes' his ; . , /1^'^xS 

' * v? £' . f ' ’1 *' », \5 .-V ,.%» ’ '<■**' V-v t «• ' 

. ‘ v ’ 1 s ' , ,« y ** , K y " 1 ^ A' V 

r f , y i' * . * *’ % v ►*'< " ,.‘i- - > 

- , - ■ ‘v % \ •>>*( * ^ - v 4 's‘\M 



to the extent that neither takes much 
representation. * 

Combination of elements . If the stock of elements for* each ‘ 

were randomly and infinitely combinable, one would need, only to iist thetotib^” £$&it 
no particular sort of ordering and with no constraints upon their occur renfiq^'* •- • ^ ? 
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One can even combine the notions of element-in-itself and its representatlbhrrJ'A'-; $ 

in minimally common features ^-so as to make the listing more parsimonious ! ’An* 

■ ■'■■*. :-v . : ■ y.v >* " 'V.<, 

Individual would thus be little more than a walking dictionary, speaking -<■’*■'• ^t 

in lists whose ordering in time is immaterial. 

It is clear, however, that this is, not the case in natural lahgtt^gh^ 

Ordering of elements in the act of speech is also requisite 
intelligibility. Ordering is also shared from individual, to indlyjldi^,l A . . 
again, such ^Ordering of elements may be. chance, total 
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convergence from individual to individual* In the natural case , the cowmuhality 

of ordering constraints in a speech community approximates totality very closely, 

but mistakes are noticed, or a speaker hearing himself may stumble, stop, and .. 

♦ „ - 
begin to speak again, correcting himself. 

From the point of view of the linguistic analysis, it may be stated that X 

the stock of elements, (b) the minimal conditions for their intelligible 

representation, and(<$ the constraints on their ordering is called the grammar 

Of the language in which two or more individuals share. The grammar is an 

abstract device, stating regularities; only, with perhaps some systematic 

^ciliary statements that account for "deviance” of certain putative sentences 

• ftom'Viormal." Vv? 

This research represents a preliminary attempt tb test the viability of th.e, 
method, to isolate the nature of ordering constraints on linguistic elements , 
when such ordering is disrupted. The results reported by no means ^fa^ to be v 
conclusive concerning the total linguistic performance of individuals; -rather, 
the analysis of the data has, as its focus for the present, questions regarding 

method. .• "''V-y . ‘V. '* •' ‘V . ■" \ t 

Fifty pairs of agrammatical statements were selected for use in this study, 

with the members of the pairs of equal syntactic length, i.e., the numbers of 
format ivea was the same for. each string in a pair. The procedures by whi qh : 
these statements were generated assured agrammaticality. Pairs were presented 
to normal speakers of English in a forced-choice, task. Ss were also required . 
to rank each statement as to its approximation to good English. ' f 
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Construction of test materials . ; A computer program was designed^ to 



generate a aet of 1000 pairs of statements of equal syntactic length. Fifty 
pairs were selected for the study. The pairs were generated from ten gram- 
matical sequences in English having the form: determiner + adverb + adjective + 
nnur, + S + verb ,+ S + noun + preposition + noun. This yields a 10 x 10 matrix, 
representing 100 grammatical English sentences > xuc prcgres operates on this 
matrix in the following fashion: 

•? 1. it first selects on a random basis, a length , between 1 and 10; 

2. it then selects, randomly, items from the matrix; 
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, The authors express gratitude to Professor B. A. Caller, University- of 
for his kind assistance in the construction of the computer program. 
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3, it continues to select items until a number of them representing 

twice the length has been reached; V 

4. it prints the selection of items with a space. between the first and 
second halves of the string generated by Step 3 above (this procedure yields 










Olitra rtf ontml • <**%A 

** — — — * 



/* f . f ' ' i 



1 a “ '' ''vZ^Z 3 



5. it continues this operation until the upper limit (1000 pairs) has 
been reached. '•'* 'A ' . V-'Z : -’v. r Z&7-1 

•. j; ; ? '.•< y ;‘r\ : , jj 



The 50 pairs (100 statements) are given in Table 1. In addition, this 
table also Identifies the pair, to which each statement belongs. in each of two 
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Insert Table 1 about here 



Y . . , ,C, 

" . * )*»-’ ' 1 ‘4 A> 



.<.** Z'Z v t* 

' A* o 

n t A 
■ *i.o. 1 






orc^rs, and the sequence in which they appear in the pair for each order../ 
Sequence of pairs was randdmized separately for the two orders . Sequence 
within the pair was also random , Each order was presented to each £ in the 
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toanher described below 
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a S« wejre randomly assigned to one of two 
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conditions , pne group of ^Ss responded to Order 1 on Day 1 of the experiiaent 

' + 1 N : ; v ‘V y ' *'* v VZ„ . * ‘ 'Vv 1 ,'Z' j 

and Order 3; on Day ,2 .^ A second group of Ss was given Order 2 on Day, 1, and 

-"J‘ 4 * ''•'>*> n ' z' . Z v ..^ ' A r ‘ /' Z:/*; >V t v:' ' 

, ^ ^dy 2* There was a ope-day interval between experimental sessions. 

- /Z. .Z^^stafementa. wdre recorded onto magnetic [ta^ for ^ presentation ^ 

Individually . .Each .pair was., preceded' by the at atement indi eating v ’* * 
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A signal which activated^ ^ a fe&her at > the end of the-, aecod^' 
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•. the second chahhjsl*.df’ the 'magnetic - tape. 

,. ^.s heard the stimuli through a loudspeaker and responded, by depressing. o^iVol^'’ 
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totts ©n n response ooarde Their choice and response latency were ' 7\'^h Jtf " ‘1 
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pupomatlcai3y recorded by a computer (PDP-4) . lipohr completion of the response 

t;he tape playback was activated and, aftef a brief internal , the next pair in 

Tv.,', v.-s?- c',rv -V‘ *\l. - A,-. • ■*.: ■■■"''■■■’>- '-A' - ’•> •' • 

the sequence was presimted All strings presented audiplly were spoken with 
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‘task idilch was^ ^ i#jinisf«f ed hamediately following ibe 'Z ” 4 
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Sivsn a, t*«t booklet which contained the 1Q0 
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preference experimeiii; , £5 

stateme^ts and a deck of cards with each statement in the ordef in which thdv 
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that ‘tape. ’./The _ response booklet had columns, marked one to five* 

v/-- -• •* •- •; - .• . • ,' -:.y '<p$£Maj 



^tdltating; apprdximation to a good English sentence, with one being completely 
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an*ca.Ft«bXe and five coopUt.iy acceptable. Ss indicsifeed thtir ranking lor 
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each statement by putting a check mark in the appropriate column* Statements 

„ - , , ' a , ( V ’ - v * ; 

were ranked on each experimental day. ^ ' . 

In the forced-choice task, Ss were told they would hear pairs of agram- 
matlcal statements which probably would not make sense. The Ss ' were 

ltl'Q f* t*f 1 ^ t* A/) I* ft mal Aaf* ♦• !%«»•* mamk a w a 4? a A.^k a a4 «* *.«U4 aU 4*1* a* t.^. i>L. i. a* 

«*•«««*• W4. OC4V4S {/«AA ,vru4»wix vilUJT a CIO* b vv WC but: UC9V 

approximation to [good] English. They were told to indicate their choice by » 

* . . . * \ * * * , - » . «* ' * , 'S • ‘ 1 * _ 

pushing one of two buttons on the response board. Ss were given as much time* 

t ? . 1 * v . - ,, *** * - w * ^ 

as they required to make a response. 

' r For the ranking task , Ss were told that they were going to rank the state-” 
ments they had just heard on the basis of their approximation to a good English 
sentence. They were instructed as to the degree of approximation represented- 
by : each number above the columns on the response sheets.; 

Subjects . Bach of 28 college students with normal hearing was assigned to 
one of two experimental conditions , Day 1, Order 1 - Day 2, Order 2, or Day 1, 
Order 2 - Day 2, Order !. ’ * , * ’ ' 
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, While all the data for this study have been obtained, the results have 
beep hhalysed and interpreted only in part. In particular, the implications . 
of the results to be described below have not yet received adequate ‘attention • 
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Results obtained for the preference task, wi.ll be taken up first . Table 2 
showsthe number of times a member of a pair was selected by the Ste for each ' 
^order of presentation on each experimental day. . , 
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Insert Table 2 about here 
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* Among the artifacts that may cohfound the present results , one is the ; 
possibility that Ss may consistently prefer one button, while another is that 

■ ( ' "; v * t ~ _ * /. t _ , - - • * * : s ' V. 

they may prefer the same statement on both experimental days. These tendencies 
could be considered either as competing, when a reversal in the sequence in the 
pair occurs oyer days, or as facilitory, when there is ho change in position. 

< ' When ali of the preference data are combined there is a significant tend-* 
«ncy toprefer the first member of the pair over the second (x * 10.0, p < .01)4 
is, however, a strong tendency for consistency in selection. For pair? 
that were not reversed across orders, therd are 454 consistent responses as 
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opposed to 209 nonconsistent responses. When order within the pair is reversed; 
there are 476 consistent responses as against 245 inconsistent responses. This 
suggests that, on the whole, Ss tend to be responding to the Statements rather 

than making some arbitrary decision without regard to the stimulus items. 

* * 

Examination of the preference data reported in Table 2 shows that there 
are considerable differences in preference among the 50 pairs of stimulus 
items. In some cases there is almost perfect agreement among Ss as to pre- 
ference. On the other hand, some Items showed little agreement among Ss. 

More detailed analysis of the individual items is necessary before any theo- 
retical and linguistic interpretation can be given to these data. 

The question of consistency of preference was also examined by obtaining 
the correlation of preference responses for orders and days. Table 3 gives 
these correlations. It is evident from the significant and high correlations 
that. Ss exhibit similar choice behavior on different days, and the order in 
which the pairs are presented does hot influence preference. 

' Insert Table 3 about here 



Table 4 shows the per<c$ant of Ss who chose one particular member of a pair 
of statements in the preference task, taken over all, conditions for all Ss . 

Insert Table 4 about here 



Latency of response in the preference task does not appear to be a powerful 

t < + *: . . , . » . j \t ' ’ < . 

variable. The amount of inter-subject variability was too great to permit any- 
thing but a gross analysis* of the data. There is a tendency for the latencies 
to increase with high inter-subject variability, v , 

The distribution of. ranks in terms of per cent Ss assigning a rank for each 
item on each day and each order is given in Table 5. Table 6 gives the distribu 
t ion of rank over Ss and conditions. Correlations between rank and preference 

’V ; : Insert Tables 5 & 6 about here 

t a ’ \ r - / - • * * ' * * . „ ’ ** V 

were obtained for each pair, The overall correlations between preference and 
rank for all four experimental conditions are given in Table 7 . These correla- 

' 't' * \ •** v ... 

K * %<r ^ < i ^ ^ 

tions are all significant at the ,01 level* This means that when s jS preferred 
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Insert Table 7 about here 

a member of a pair, he tended to give it a high rank. When the member of a pair 
was not preferred it was given a low rank. These findings lend support to the 
argument that the j> was in fact using his knowledge of English to guide him In 
his preferences, as well as his decision with respect to an item's approxima- 
tion to good English. 

S.8 are remarkably consistent in their ranking behavior on this task, as is 
illustrated by the correlations given in Table 8. These correlations are all 

Insert Table 8 about here 

significant at the .01 level; there is no evidence to indicate that rankings 
tend to be higher or lower in the initial or final portions of the set of items. 
Analysis of variance was carried out to determine if there were significant day, 
list or order effects which were influencing the results. The obtained F 
values did not approach significance. 

If k / * * • ; *’ ’ ’ 1 " - ' 1. 

Discussion ' 

Given a stimulus which deviates from normal English, presented to a native 
speaker of English, one might expect the following: 

1> abizarreness reaction to any (putative) sentence which deviates in any 
way from complete normality; 

2. less of a bizarreness reaction to one putative sentence than to another 
(i.e., a preference, under, appropriately clear instructions in experimental 

situations), if the one approximates more tlosely a "normal" sentence than 

*> ' 

does the other; 

3. to the extent of the deviance from normal "sentencehood* *' uneven 

, « ' t * . 1 * 4 

bizarre reactions to putative sentences, though all strings presented may • 
quite deviant, e.g. , t 

*Make delicious eats. ; ‘ 

*Rice above a. • ‘‘ 

4. that the Individual to whom a putative sentence, or sentences, is ■ 

^ s 1 , * ' 

ptes anted will either assign a higher rank to, or express preference for; 
sentences which in one way or another approximate "first-order grammaticalness" 
more closely than others. 



Tikofsky ° 

Functionally* it Is not assumed that the grammar Is reified, except for 
purposes of ease in discussion; and further it is not assumed that ’gram- 
m^ticalness" is in (or partially in) any sentence or putative sentence, hut 
rather is the description of the normality assigned to it by the members of 
the speech community which utilize it for communication. Thus, one of the 
goals of the present study is to make intersub j ective what is at present 
intuitive. Two sets of possibilities, each generable from differing view** 
points, serve as a basis for interpretation of the obtained data. 

When a given individual makes the response "closer to normal" to a given 
string or putative sentence, that individual is contributing criteria of 
judgment which represent his own (learned) abilities to Interpret and respond 
to sttrings of formatives in his language. This is insured by the method in 
which the stimulus materials were constructed: all Surface syntactic constraints 
were removed. The response "closer to normal," behaviorally, may ^rise from 
one or both of two possible properties of the (normal adult) organism:.: 

(a) the string possesses, for him, a greater degree of situational adequacy 
than others (i.e., it is semantically closer to normal English) ; or (b) in 
the absence of this condition, the string possesses a greater degree of gram- 
matical .adequacy than others (i.e., it is formally closer to normal English 
?&ftaps in several respects). ' ' , 

However, since interpretations like the ones above are basically 
competence-considerations, they must retain the status of hypotheses and ,, 
await further refinement of our understanding, both of language and of the 
organisms using it. And since the act of interpretation itself is a per- 
formance-consideration, having all the typical constraints of performance 
(limited memory, experience, history, etc.), we must say that our data are 
performance data and their configurations are performance-configurations. 

Iii this way, it may be concluded that when an individual states that he 
prefers one string to another, he is in effect stating that either (a) he 
hM previously responded to the string elsewhere, or to one similar to it, 
regardless, of its degree or type of deviance; or (b) if this is not true, 

. the preferred string is more easily "adjustable," or interpretable, than ate 
others via projections made on it from the S/s normal linguistic repertoire. 
What cannot be shown by these data, of course, are theoretically-expressible 
properties of strings whose analogues are not within the individual jS’s 
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Another question which lies within the domain of performance-criteria is 

A ’ * 

the following: does the £ express preference for a string on the basis of only 
a portion of it? At least, in one sense, there is a possibility that given two 
nonsensical, agrammatlcal strings, the one which possesses even the slightest 
suggestion of "normality , " as opposed to the other, will be chosen, since the 
choice is forced. For example, sentences 5 and 6, presented (iiudiolly) as a 

pair, are as follows: . 

5. "Meal sand soldier wine after the." 

6, "Show rice wine daughters delicious home." 

Categorical relations for English determine that the only reasonable projection 
on sentence 5, for normality of element-ordering, is on the substring "after 
the." The remaining elements of the statement can be treated as a 1. 't. How- 
ever, in sentence 6, there are many features of the structure of English, and N 
perhaps, also features of a given jS’s experience, which might be projected upon 
the string. First, in terms of an individual’s experience, the string might 
appear to have "telegraphic" form, i.e., determiners have been deleted for 
economy. On the other hand y it resembles, in its syntactic shape, either an 
imperative or an infinitive construct : something similar, say, to 

6a « "Show high society women beautiful jewelry" [in answer, perhaps to 
the question, "What does George do for a living?"]**, "What are my duties?" 
in which case it resembles a truncated sentence; or perhaps it might be an 
imperative [in answer to the question, "What shall 1 do today?"]. , 

The relations between show — home , or show — daughters— home are perfectly 
normal sequences which can occur in a number of fully adequate sentences in 

English. The sequence rice wine is normal. Further, though grammatical 

* 4 / , , k 

relations are obscure, the collocation of wine — delicious is common. 

finally $-.<• In sentence 5, the only possible collocation of semantic items 
might Involve meal — wine— af ter— the ; however, their ordering is far from 
normal , and th? particular ordering would not appear in normal English even 
under the most complex of .transformational relations, and even when these are 
so used as to produce that sort of deviance found in use as a literary device. 

This interpretation is not Intended to explain the fact that sentence 6 
was preferred to sentence 5 by more than 75 per cent of the experimental 
population. It simply serves to suggest that sentence 6 appears to accommodate 
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more projections by S, from a grammatical pattern that he doubtless knows. In 
addition, sentence 6 exhibits a larger number of compatible semantic notions - 
or situational responses, and has more parts which,, to the exclusion of the 
rest of the string, are either fully normal or closer to normal than anything ; 
found in sentence 5, i.e., are constituents of fully-grammatical and meaningful 
English sentences* While these present data do not yield discrlminable inf or*- 
mation as to whether the Ss* choices were made on the basis of semantic/ 
situational adequacy, lexical adequacy, or syntactic adequacy. Studies are 
now in progress in which it is hoped this discriminability will be obtained. 

We believe that for some reason or other, native speakers of a language . 
do impose interpretations upon random strings of formatives in their language . 
Ss do this even when it is known that there are no constraints whatsoever 
within the strings themselves (i.e., even when the ’’sender” is encoding no 
message, the "receiver” does not receive a non-message, but, rather, attempts 
to reconstruct some sort of message). Further, this is done not only on the 
phonetic and phonological levels (where much research has been done to 
demonstrate this) but also, as is shown here, on levels that can only be 
called quite abstract: syntax, lexicon, meaning. 

Recent claims, on the theoretical side, to the effect that an organism 
acquires a grammar and that at some point in its history it can then interpret- 
er impose Interpretations on — any string in the language whose grammar has been 
internalized, appear to be supported at least in part by the results of the 
present investigation. Most significant is the fact that the Ss in this 
research were not trained to their experimental task, nor were they conditioned 
to some notion of "acceptability” (which in terms of the goals here would 
simply beg the question: how does the experimenter know what is "acceptable?”) , 
nor was there any pattern in the stimulus materials to which they might 
ultimately have come to respond during the experimental sessions. In short, 
what has been demonstrated, experimentally, is something contributed by the 
individual to putative linguistic materials which contributed nothing in them- 
selves. If the choices are considered as dependent variables, then we have 
shown only that there are indeed independent variables controlling the choices. 
The extent, nature, and function of these variables will be the subject of 
further research. 
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Tibltl 

Stleulus SUtomti, Order of and 

lute on Awlfmmt for Order* X end 2 



Statement 



1. Pews fake hone book old profceeor* no. 
2* With drink the young knowledge print* 
ly Mpielilly anesgs. 

3. Whlskaye straws wine snakes daughters. 
k'j Young young bulldlnga grow leas hols 
. Milk.' 

S. Heel tend soldier wine after the. 

I. Shaw rie* win* daughter* delicious 
bout. 

7. M*w show those extreuely. 

I. Klees above a* 

9* : Intelligence news good dellelouaea 
. tasny strong. 

10. Ixtreel; priatnrily beautiful foods 

food windows lakes. 

11. the allk thoae. y<ec . 

12. Around silk paint around. 

13. Around drink machines windows nor* 

book the jobs. 

14. Many especially a big ugly auchlnteoaoe. 

15. Help qultesss sxtreeely windows paint 

lady sand. 

14. During sand paint asseotlals good sen 
above big. 

17. Very*. 

18. Make anong. 

19. Croup dellelouaea paint sons outside. 

20. Lady sand .through grow eoldlers. 

21. Hose dishes. 

22. Eat Si 

23. Food thoae during woods thoaes. 

24. Old rlca good delicious grow strong sat. 
2$. With professors lakas drink. 

26. Sevan seven nan seven :eachinee view. 

27. Make delicious tats. 

28. Through profsr few delicious. 

29. Professors group group. 

30. Quit* dlshta seal after. 

31. Food show. 

32. Above wine. 

33. Han grova soldiers aeong ssvsral. 

34. lielp view daughtsra ins whiskey dishss. 

35. Eon* oak* rices showssaas intelligence. 

36. Quite Jobs strong uachlnes very drink* 

strew help. 

37. Very with sake extreesly the wlndovaes. 

38. Hawaaa vary delicious nan. 

39. Crow makes rice. 

40. building quite eoldlera. 

41. Especially old. 

42. lekeaes. * 

43. Klees greens grow view. 

44. Around above lady like fields lady. 

65. look bigs during grten prefer jobs**. 

46. Frefare daughters groups the drinks. 

47. Weeds. 

48* Machine* else oat. 

49* With oettidt alow clooeao help. 

30. Classes nachinee building Intelligsace 
•specially. 



Ordsr 1 Order 2 



Pair Wo. 


Button 


Pair Ko. 


button 


1 


A 


38 


8 


1 


• 


38 


A 


2 


A 


22 


A 


2 


■ 


22 


8 


3 


4 


44 


A 


3 


8 


44 


• I 


4 


A 


11 


8 


4 


B 


11 


A 


5 


A 


36 


8 


5 


• 


36 


A 


6 


A 


4 


.9 


6 


8 


4 


A 


7 


A 


19 


A 


7 


8 


19 


B 


8 


A 


21 


9 


8 


B 


' 21 


A 


9 


A 


45 


9 


9 


B 


45 


A 


10 


A 


12 


b 


10 


B 


12 


A 


11 


A 


26 


A 


11 


B 


26 


B 


12 


A 


6 


S 


12 


8 - 


6 


A 


13 


A 


3 


B 


13 


B 


3 


A 


14 


A 


37 


B 


14 


8 


37 


A 


IS 


A 


7 


B 


15 


B 


7 


A 


16 


A 


14 


B 


16 


B 


14 


A 


17 


A 


10 


A 


17 


9 


10 


b 


18 


A 


32 


B 


18 


8 


32 


A 


19 


A 


43 


A 


19 


B 


43 


B 


20 


A 


8 


A 


20 


8 


8 


B 


21 


A 


5 


A 


21 


8 


5 


B 


22 


A 


16 


B 


22 


8 


16 


A 


23 


A 


23 


A 


23 


8 


23 


1 


24 


A 


20 


B 


24 


8 


20 


A 


25 


A 


31 


B - 


23 


8 


31 


A 
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. Tabt« I (Coattnud) 

ftieulua Statements. Ortaf of Presentation 'and 
Buttoa Assignment for Ordars 1 and 2 



Stusaaent Order 1 Order 2 



> 


Pair 


Ho. Button 


Pair No.. 


Button 


51. Soldier group naktiti wjoi oao diehsbas. 


26 


A 


34 


A 


52. Siaply paint uke with these daughters 


26 


B 


34 


8 


MWft fiPMia. 

m— — C ** 


53. Hose ateei classes. 


27 


A 


2 


A 


54. Beautifuls entree*!?. 


27 


I 


..2 


» 


55. This book daughters. 


28 


A 


4'<» 


A 


56. takes new totally. 


36 


1 


46 


B 


37* Soscaea Ilka lakes few windows ugly. 


29 


A 


30 


B 


50. Pood strong eiaplyeeecseseess. 


29 


71 ' *• 


30 


A 


55. Strong very through grows « jobs vary. 


30 


A s t 


4X 


A 


60. Outside very leas sand help less strong 
ugly. \ v " 


30 


8 


41 


S 


61. The beyond knowledge those# building 
.'•wtl Intelligence. 


31 


A 


40 


A 


62. Wlndowsea hole# grounds help ptriaerlly. 


31 


1 .* 


40 


1 


63. feint especially*. 


32 


A 


9 * 


A 


64. * Essential* dishes Jobs. , - 


32 


B 


9 


' B 


65. After totally. 


33 


A 


1 


A 


66.. Buildings. 


33 


B 


' 1 


B 


67. Windows Intelligences strong soldier 


34 


A 


13 


A 


•even hole intelligence. 


68. Aftsr -see* especially help lady daughters 
new send. 


34 


B 


13 


B 


•69. Big hose. 


35 


A 


24 


A 


70; Those several. * 


35 


1 


24. 


B 


71. Beyond primarily seven fields this. 


36 


A 


*• 48 


B 


72. Prefer intelligence grow big seeks. 


36 


8 


48 


A 


73; Vldw group intelligences. 


37 


A 


35 


A 


74. Beautifuls young nan.. 


37 


* B 


35 


B 


75. Especially a sand helps these extresely 


38 


A 


25 


. B 


•* view the. 


76. Leases cat totally thosss with beautiful 
theaa. 


38 


B 


25 


A 


77. . Prefer jobs ieaa nan. 


39 


A 


17 


A 


76. Green pvUuliy Specially wood. 


39 


8 ‘ 


17 


' B 


79. Several rice during book outside hole. 


40 


A 


15 


A 


80. quite throughece prefer daughters. 


40 


B 


. 15 


B 


61. Help outside whiskey profescor good 


41 


A 


50 


A 


entreaelye with. 


62. More auoaga classes wlndowsea above big. 


41 


B 


50 


B 


63. Steel fields knowledge .eat wood. 


42 


A 


39 


A 


8i\. In shows seven around. * 


42 


B 


39 


« 


65. Prefer buidling daughters seal classes 
few withe nasi. 


43 


A 


42 


B 


86. Whiskeys fcuildlngsesss Jobaes daughters. 


43 


B 


42 


A 


67. Claeses beyond with green. 


44 


A 


18 


• 

as 


Si. Machines seres the hoee. 


44 


B 


18 


A 


89. Ugiya prefer good a these dsliclouseses. 


4’> 


A 


33 


1 


W. Pew beyoad fields aleply book after big 


45 


B 


33 


A 


holt. 


91. After -help nachinc'a dlahtses ante 


46 


A 


49 


B 


boildingc. 


92. Abovtt solditr i«vteil group professor 


46 


B 


49 


A 


foods ugly. 


93. Group aevan; aleply green. 


47 


A 


29 


A 


94. Above biga heal. 


47 


B 


29 


B 


95. Show after oaveral help profaeaore above. 


41 


A 


27 


A 


96. Seven profeaeMa good view. 


48 


• 


27 


B 


97. Varya. 


49 


A 


28 


B 


96. With ulna* 


49 


B 


28 


A 


99 * Several tbeaaa prefer eat group aiaply 


50 


A 


47 


B 


aleply. 


100* Street the aaal drink War lag *wite classes. 


SO 


B 


47 


A 
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Tekl« 2 

Nurixr of TIim a Nwktir of a Pair Selected by ji» for Days 
«ti Orders «o£ for Orfna Hem* Cays. 0*«14, io issb Samp) 



Seataatat 


Bay. 


Day. 


o*y a 


Oay- 


Order | 


Orderj 


0rdar| 


Order, 


Ordetg. 


QrderJ 


I. Paws Mk* hesebsvk old profeeaora ou. 


6 


10 


10 


9 


15 


20 

■ 


9 . U< •>?:■() »Ka mum kaauleiM IlflnarlT 


8 


4 


4 


5 


13 


0 


especially osonge. ^ 


3* Whiskeys straws wtos sabss dauahtsre. 


7 


4 


1 


z 




3 

A 


4* Yakuts yeung buildings grow lass Koto 


7 


IO 


13 


12 


15 


22 


•ilk. _ , 


5. Hast sand sold tar wins sftar tbs. 


2 


2 


1 


7 


9 


J 


4. Show rica wins daughters delicious 


12 


12 


13 


7 


19_ 


25 


hosa. 

T. Haw show thasa extraaely. 


12 


11 


11 


10 


22 


22 


8. Sicas above a. 


2 


3 


3 


4 


6 


6 

dk 


9. Intallifanca nawa good dalle iouaeo 


5 


7 


5 


4 


9 


12 


■any strong. 

10. Extrtoely primarily boautlful foods 


9 


7 


9 


10 


19 


16 


food windows lakas. 


11. The Bilk those rice. 


4 


5 


1 


4 


s 


ft 


12. Around milk paint around." 


10 


9 


13 


10 


20 


22 


13. Around drink sachlnes windows note 


book the Jobs. 


12 


13 


13 


10 


22 


26 


14. Many especially a big ugly aachlneseoes. 


* 


1 


1 


4 


6 


2 


15. Help quiteses extrenely windows paint 


5 


5** 


9 


10 


t t 


u 


ledy sand. 


16. During sand paint essentials good nan 


9 


8** 


5 


4 


13 


13 


above big. 


17. Varya. 


7 ** 


4 


5 


♦ 5 


ii 


9 


18. Maks anong. 


6** 


10 


9 


9 


15 


19 


19. Croup dellciouaea paint none outside* 


8 


5 


8 


3 


11 


13 


20. lady sand through grow soldi arc . 


6 


9 


6 


. 11 


17 


15 


21. Hons dishes. 


10 


9 


7 


8 


18 


16 


22. Eats. 


4 


5 


7 


6 


10 


12 


23. Pood those during woods thoses. 


2 


3** 


5 


4 


6 


8 


24. Old rice good delicious grow strong ast. 


12 


10** 


9 


10 


22 


19 


25. With professors lakes drink. 


7 


5 ** 


1* 


10 


17 


6 


26. Sevan seven un seven nachinas view. 


7 


6** 


12* 


• 4 


11 


18 


27. Make delicious eats. 


10 


10 


10 


12 


22 


20 


28. Through prefer few delicious. 


4 


4 


4 


2 


6 


8 


29. Professors group group. 


9 


7 


12 


11 


20 


19 


30. Quits dishes neal sftar. 


5 


7 


2 


3 


8 


9 


31. Food show. 


3 


2 


5 


6 


9 


7 


32. Above wins. 


11 


12 


9 


8 


19 


21 


33. Han grows soldiers aaong several. 


13 


11 


14 


11 


. 24 


25 


34. Help view daughters ins whiskey dishes. 


1 


3 


0 


3 


4 


3 


35. Hose sake rices showseses Intelligence. 


4 


6 


8 


9 


13 


14 


36. Quite Jobs strong nachinas vary drinks 


10 


8 


4 


5 


15 


14 


st?av help. 


37. Very with sake extremely the vlndoweee* 


2 


6 


8 


8 


10 


14 


38. Mewses vary delicious nan. 


12 


8 


6 


6 


18 


14 


39. Crow aakes rice. 


10 


12 


11 


12 


22 


23 


40. lulldlng quits soldiers. 


4 


2 


3 


2 


6 


5 


41. tapedally old. 


13 


12 


14 


12 


25 


26 


42. Lakeaes. 


1 


2 


0 


.. 2 


3 


2 


43., Rlcss gresns grow view. 


6 


10 


9 


9 


15 


19 


44. Around above lady like fields lady. 


8 


4 


5 


5 


13 


9 


45. look bigs during green prafar Jobasa. 


1 


7 


2 


5 


6 


9 


44. Prefers daughters groups the drinks. 


13 


7 


12 , 


9 


22 


19 


47. tfooda. 


12 


8 


10 


6* 


18 


18 


48. Machines clasaaa. 


2 


6 


4 


7* 


9 


10 


49. With outside show classes help. 


9 


14 


13 


11 


20 


27 


SO. Classes sechirn building intelligeece 


3 


0 


1 


3 


8 


1 


. especially 


31. Soldier jtet-v aakacos wood ms diabases 


2 


5 


3 


2 


4 


8 


32. Singly paint asks with thasa daughters 


12 


9 


11 


12 


24 


20 


mmm |tMf« 


M. Hobs gtasi clssssa. 


10 


12 


13 


14 


24 


25 
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Table 2 (Coetinuad) 

tkabtr of Ti«M a Havif of a fair Selected by Ss for Day* 
and Orders ssi for Orders Across Days. (M*14, in lach Group) 



Stateneot 


o«y» 


Oay. 


Day- 


Bay- 


Orderg 


Order: 


Order£ 


Order j 


Order^ 


54. Beautifula extreeely. 


4 


2 


1 


0 


4 


55. This boob daughters. 


5 


9 

dt 


5 

m 


6 

• 


11 

2? 


56. Lakes aw totally. 


'W 


X 


9 


w 


d* T 


57. Sonesee l£ks lakes few siadows ugly. 


13 


Vk 


n 


14 


V 


58. food stroag elsyly sees aSssses . 


1 


0 


i 


0 

a 


1 

17 

a a 


59, Stroag eery through gross s Jobe very. 


8 


6 


8 


9 


60. Outside vary leas scad help less stroag 


6 


8 ' 


6 


s 


11 


ugly. 

61. The beyond knowledge thoses building 


10 


7 


S 


* 


16 


saved intelligence. . 


62. Vlndowses holes around* help primarily. 


a 


7 


6 






63. Palat espelciallys. 


6 


7 


7 


s 


11 
a 4 


64. Eseeatlals dishes jobs. 


8 


7 


7 


9 


17 


65. After totally. * 


3 


10 


4 


* 


12 


66. luildingS: 


11 


3 


10 


5 


16 


6?. Windows Intelligences, strong soldier 


X 


4 


4 


2 


A 


. seven haU totel!«i«nce. 

68. After sons especially help lady daughters 10 


10 


10 


12 


12 


new sand. 
49. 81g hose. 


12 


12 


13 


11 


21 


70. Those several. 


2 


2 


1 


3 


S 


71. Seyond prlaarlly eaves fields this. 


6 


6 


8 


5 


11 


72. Prefer Intelligence grow big sake. 


8 


s 


6 


9 


12 


73. View group intelligences. 


9 


11 


12 


10 


19 


74. Beautifuls young nan. 


5 


3 


2 


4 


9 


75. Especially a saud helps theae extreealy 


4 


13 


7 


7 


11 


view the. 

’6. Lessas cat totally thosas with beautiful 


10 


1 


7 


7 


17 


theae. 


77. Prefer Jobs less nan. 


5* 


7 


9 


4* 


11 


78. Green prlaarlly especially wood. 


7 * 


7 


5 


4* 


11 


79. Several rice during book outside hole. 


7 


7 


6 


5 


12 


80. Quite threughwes prefer daughters. 


7 


.7 


8 


9 


16 


81. Help outside whiskey professor good 


10 


9 


7 


9 


19 


extreeelye with. 


82. More aaongs c lessee vlndowses shove big. 


4 


6 


7 




V 


83. Steel fields knowledge eat wood. 


6 


4 


3 


5 


11 


84. In shows seven around. 


8 


10 


11 


9 


17 


85. Prefer buidling daughter, seal classes 


12 


7 


9 


11 


23 


withe seal. 


86. Whiskeys- bulldingeeaea Jobses daughters. 


2 


7 


5 


5 


5 


87. Classes beyond with green. 


6 


7 


8 


6 


12 


88. Machines seven the hone. 


8 


7 


6 


8 


16 


89. Uglys prefer good e these dellclouseses. 


2 


4 


5 


3 


5 


90. Pew beyond fluids elaply book after big 


12 


10 


9 


11 


23 


hole. 

91. After help aachlnee diaheses sores 


9 


5 


8 


5 


14 


buildings. 


92. Abovea soldier several group professor 


5 


9 


6 


9 


la 


foods ugly. 


93. Group aewon elaply green. 


13 


13 


12 


13 


26 


96. Above bigo seal. 


1 


1 


2 


1 


2 


95. Show after several help professors shows 


. 10 


9 


7 


7 


17 


96. Sevas pxefersssss good view. 


4 


5 


7 


7 


11 


97. Varya. 


3 


4 


1 


3 


6 


98. With wine. 


11 


10 


13 


11 


22 


99. Several theses prefer ant group ainply 


7 


6 


10 


4 


11 


elaply. 

100, Strcag the seal drink during guite 


7 


1 


4 


10 


17 


classes**. 


Aftatrioe with * will sot add up to 14 because 
necessary to reject sons of the response*. 
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14 

|4 

27 
* I 
14 
14 

i> 

13 

14 
14 
14 

13 
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20 

25 
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14 

14 
23 

5 

20 
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16 

12 

13 

15 

15 

13 

7 

21 

16 

12 

15 

13 
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19 

13 

15 

25 
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16 
12 

5 

23 

14 

12 
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Table 3 

Correlation Coefficients to Estimate Degree 
of Preference Consistency for Orders and Days 



Condition 



Day 1 - Order 1 and Order 2 

Day 2 - Order 2 and Order 1 

Order 1 ~ Day i and Day 2 

Order 2 - Day 1 and Day 2 




.639 
.723 
.888 
.781 . 



*all r’s significant at or beyond the .01 level of confidence 
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Table A 

Per Cent Ss Preferring Statements Presented 
in Pairs in the Preference Task 



' M ** *' sU i > * >,1 &*' % ^ 

* ?* W \ V* j^I^y * 

’ V'' V 

■*• ; - ,:. - -\. 

■ • ' ... '• v^A'V 

Y * -Y YY"‘ 

• ., l6 /...>.•■ .-;v ; ; 

■' *ri* , 



0-24% Preference 



No. 



statement 



2. With drink the young knowledge primarily especially amongs. 

3. Whiskeys straws wine makes daughters. 

5. Heal sand soldier wine after the. 

8. Rices above a. 

14. Many especiallys big ugly machineseses. 

34. Help view daughters ins whiskey dishes. 

40. Building quite soldiers. 

42 . Lakeses . 

50. ’ Classes machines building intelligence especially. 

51. Soldier group makeses wood man disheses. 

54. Beautifuls extremely. 

58. Fc^d strong simplyseseseseses. 

70. Those several. 

94. Above bigs meal. 

97 . Verys . 

* 25-49S Preference 



9. Intelligence news good deliciouses many strong. 

11. The milk those rice. 

16. Durings sand paint essentials good man above big. 

17. Verys. 

19. Group deliciouses paint some outside. 

22 • Eats . 

23. Food those during woods thoses. 

25. j With professors lakes drink. 

28. •> Through prefer few delicious. 

30. Quite dishes meal after. 

31. Food show. 

35. Home make rices showseses intelligence. 

37. Very with make extremely the windowses. 

44. Around above lady like fields lady. 

45. Book bigs during green prefer jobses. 

48. Machines classes. 

55 . This book daughters • 

60. Outside very less sand help less strong ugly. 

62. Windowses holes arounds help primarily, 

63. Paint especially s. 

65. After totally. 

67. Windows intelligences strong soldier seven hole intelligence. 

71. Beyond primarily seven fields this. 

74. Beautifuls young man. 

76. Lesses cat totally thoses with beautiful these. 
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Table A (Continued) 

Per Cent Ss Preferring Statements Presented 
in Pairs in the Preference Task 

25-49% Preference - cont T d 

No . Statement 

77 . Prefer jobs less man • 

78. Green primarily especially wood. 

79. Several rice during book outside hole. 

82. More amongs classes windowses above big. 

83. Steel fields knowledge eat wood. 

86* Whiskeys buildingseses jobses daughters . 

87. Classes beyond with green. 

89. Uglys prefer good a these deliciouseses . 

91. After help machines disheses mores buildings. 

96. Seven preferseses good view. 

99. Several theses prefer eat group simply simply. 

50-74% Preference 

1. Fews make home book old professors man. 

10. Extremely primarily beautiful foods food windows lakes. 
15. Help quiteses extremely windows paint lady sand. 

18. Make among. 

20. Lady sand through grow soldiers. 

21 . Home dishes . 

24. Old rice good delicious grow strong eat. 

26. Seven seven man seven machines View. 

29. - Professors group group. 

32. Above wine. 

3$. Quite jobs strong machines very drinks straw help. 

38. Newses very delicious man. 

43. Rices greens grow view. 

46. Prefers daughters groups the drinks. 

47 . Woods • 

56. Lakes new totally. 

59. Strong very through grows a jobs very. 

61. The beyond knowledge thosei building seven intelligence. 
64. Essentials dishes jobs. 

66 . Buildings . 

72. Prefer intelligence grow big make. 

75. Especially a sand helps these extremely view the. 

80. Quite throughses prefer daughters. 

81. Help outside whiskey professor good extremelys with. 
•84„ In shows seven around. 

85. Prefer building daughters meal classes few withs meal. 
,88. Machines seven the home. 

92, Aboves soldier several group professor foods ugly. 

95. Show after several help professors above. 

100. Strong the meal drink during quite dasseses. 
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* t 

;18 

Table 4 (Continued) f . 

Per Cent Ss Pref erring Statements Presented 
in Pairs in the Preference Task 

75% or Greater Preference 

No. Statement 

4. Young young buildings grow less hole milk. 

6. Show rice wine daughters delicious home. 

7. New show these extremely. 

12. Around milk paint around. 

13. Around dri&k machines windows more book the jobs. 

27. Make delicious eats. 

33. Man grows soldiers among several. 

39. Grow makes rice. 

41. Especially old. 

49, With outside show classes help. 

52. Simply paint make with these daughters news group. 

53. Home steel classes. 

57. Someses like lakes few windows ugly. 

68. After some especially help lady daughters new sand. 

69. Big home. 

73.* View group intelligences. 

90. Few beyond fields simply book after big hole. 

93. Group seven simply green. 

98. With wine. 
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Table 6 (Continued) 

Per Cent Cs Assigning Ranks to Each Statement 
Summarized for All Conditions (N*28) 
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Table 6 (Continued) 

Per Cent j5s Assigning Ranks to Each Statement 
Summarized for All Conditions (N*28) 
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Table 7 

Overfill Correlations Between Preference and Rank 
for the Four Experimental Conditions 



Condition 




Day 1 - Order 1 
Day 2 - Order 1 
Day 1 - Order 2 
Day 2 - Order 2 



.681 

.804 

.444 



.622 



' *all r’s significant at or beyond the .01 level of confidence 



Table 8 



Correlations Between Ranks for Days and Orders 



Condition 



Day 1 - Order 1 and Order 2 

Day 2 - Order 1 and Order 2 

Order 1 - Day 1 and Day 2 

Order 2 ** Day 1 and Day 2 



,753 

.669 

.820 

.699 



*all r’s significant at or beyond the .01 level of confidence 



